The Effect of Contention on the Scalability of Page-Based Software Shared Memory Systems

In this paper, we examine the causes and effects of contention for shared data access in parallel programs running on a software distributed shared memory (DSM) system. Specifically, we experiment on two widely-used, pagebased protocols, Princeton’s home-based lazy release consistency (HLRC) and TreadMarks. For most of our programs, these protocols were equally affected by latency increases caused by contention and achieved similar performance. Where they differ significantly, HLRC’s ability to manually eliminate load imbalance was the largest factor accounting for the difference. Finally, to quantify the effects of contention we either modified the application to eliminate the cause of the contention or modified the underlying protocol to efficiently handle it. Overall, we find that contention has profound effects on performance: eliminating contention reduced execution time by 64% in the most extreme case, even at the relatively modest scale of 32 nodes that we consider in this paper.